The present invention relates generally to the processing of handwritten information and, more particularly, to systems, computer program products and methods for transmitting and processing handwritten data.
A variety of devices have recently been developed to capture and process handwritten information. For example, devices have been developed to capture handwritten information and to convert the handwritten information into text utilizing various character recognition techniques. This text can then be further processed, transmitted or otherwise utilized in the same fashion as text that has been entered via a keyboard or the like. In addition, devices have been developed that capture handwritten information and thereafter maintain the handwritten information in handwritten form for subsequent transmission, display or the like. For example, the handwritten information may be a map, a diagram or other drawing, or a document that includes Asian language characters that are not easily entered using standard keyboards. As such, these devices generally maintain an image of the handwritten information for subsequent transmission or display.
The devices that capture handwritten information are computing devices such as a pen-enabled computing device. A pen-enabled computing device enables the user to input handwritten information with a writing stylus instead of a mouse or a keyboard. In this regard, a pen-enabled computing device includes a handwriting capture interface. The handwriting capture interface may be a touch-sensitive screen and an associated processor for determining the coordinates that have been contacted by a writing stylus. Alternatively, the handwriting capture interface may include an electronic writing tablet having an overlaid writing surface. The writing surface may include, for example, a blank sheet of paper or a preprinted form. By writing upon the writing surface by means of a writing stylus, the user generates a visible, or xe2x80x9cwritten ink,xe2x80x9d copy of the handwritten information. The handwriting capture interface of this exemplary pen-enabled computing device also includes a processor for detecting movement of the writing stylus upon the electronic writing tablet, i.e., relative to the writing surface, in order to capture an xe2x80x9celectronicxe2x80x9d copy of the handwritten information. Depending upon the design of the handwriting capture interface, the writing stylus may be a conventional pen or pencil, and the stylus may have a radio transmitter, a camera, an ultrasound transducer, any of a variety of magnetic or electric field devices, or any combinations thereof.
As will be apparent, the pen-enabled computing device may be capable of producing both an electronic ink and a written ink copy of the handwritten information. Since the writing stylus is generally capable of providing a written ink copy of the handwritten information on the writing surface, the user is automatically provided with a hard copy, or visual feedback. It is understood, however, that a written ink copy of the handwritten information may not be provided in some instances, such as those instances in which the handwritten information is only captured and stored by the pen-enabled computing device as an electronic copy.
Typically, a pen-enabled computing device senses the position and/or movement of the writing stylus with respect to the electronic writing tablet. Coordinates, such as the X and Y coordinates, representative of the position of the writing stylus at different points in time are stored by the pen-enabled computing device as the electronic copy of the handwritten data. The handwritten data is then often used for other purposes, such as by being incorporated into documents or being displayed. In order for the handwritten data to be subsequently utilized, it is sometimes translated from the handwritten form to text form. While text translation schemes may facilitate practical uses for the handwritten data, they are often not able to accurately translate the handwritten information. In addition, translation routines may impose additional processing and storage requirements that may disadvantageously add to the size and cost of the pen-enabled computing device. Moreover, certain handwritten information, such as maps, diagrams or other drawings, and documents containing Asian language characters, is highly usable in image form. Thus, the demand for pen-enabled computing devices that are capable of utilizing handwritten information for subsequent purposes, such as processing, transmission or the like, without requiring the handwritten information to first be translated into text form is increasing.
In a number of applications, the handwritten information that is captured by a pen-enabled computing device is designed to be transmitted to another, remotely located computing device. For example, a user may wish to provide a friend or co-worker in another town with a hand-drawn map. In these applications, the pen-enabled computing device is therefore one component of a larger system or network which permits the handwritten information that has been captured by the pen-enabled computing device to be transmitted to another remote computing device. In this situation, the pen-enabled computing device can include a transmitter for transmitting the handwritten data to a server, either via the Internet, wirelessly or according to some other transmission medium. The server then further distributes the handwritten data to the remote computing device, also via the Internet, wirelessly or according to some other transmission medium. The recipient can then display or otherwise process the handwritten data that has been received by the remote computing device.
Systems that include pen-enabled computing devices are desirably designed such that the resulting image of the handwritten information that is displayed for the recipient by the remote computing device accurately reflects the original handwritten information that was entered by the user and does so in a manner that is aesthetically pleasing. As such, the processor of some conventional pen-enabled computing devices smoothes the handwritten information prior to transmission of the handwritten information. As such, the image of the handwritten information that is eventually displayed to the recipient will appear to be less jagged and will appear to more accurately represent the handwritten information originally entered by the user. In order to smooth the handwritten information, the processor initially captures a plurality of coordinates representative of the position of the writing stylus relative to the electronic writing table at different instances in time, thereby defining one or more writing strokes. In order to display the handwritten information, the coordinate data is rendered into an image by utilizing line-drawing techniques known to those skilled in the art. The resulting image can be smoothed, such as by applying antialiasing techniques known to those skilled in the art. In this regard, antialiasing techniques adjust the gray-scale values of the pixels of the image in the vicinity of the lines such that the resulting lines appear to be smoother and have fewer sharp edges or corners. By way of example, the gray-scale values of the pixels along a line perpendicular to the line constructed by the line-drawing techniques can be modified to create a relatively smoothly shaped profile, as opposed to the relatively rectangular profile otherwise created by the line-drawing techniques. By repeating this process at each of a plurality of points along the line constructed by the line-drawing techniques, the image of the handwritten information can be smoothed.
Sometimes, the resolution of the handwritten information will also need to be changed from the resolution at which the handwritten information was captured to a new resolution at which an image of the handwritten information will be subsequently displayed. In instances in which it would be desirable to increase the resolution of the handwritten information, such as in instances in which the image of the handwritten information will be displayed in an enlarged manner by the remote computing device, a pen-enabled computing device may further process the handwritten information prior to its transmission to the remote computing device such that the resulting image of the handwritten information displayed by the remote computing device has a resolution that equals or exceeds the resolution at which the handwritten information was originally captured. In order to improve the resolution of the handwritten information, the processor of the pen-enabled computing device can create new points in the vicinity of the coordinates that define the handwritten information. For example, the processor can create new points in the vicinity of adjacent coordinates by means of known interpolation, approximation or curve-fitting techniques, such as cubic polynomial interpolation, approximation or curve-fitting, in order to effectively create additional coordinates near each original pair of adjacent coordinates. The image of the handwritten information provided to the remote computing device is therefore based not only upon the original coordinates captured by the handwriting capture interface, but also upon the additional coordinates created by interpolation, approximation or curve-fitting. As such, the resolution of the resulting image is effectively enhanced.
As will be apparent, the smoothing of the handwritten information and the creation of new points in the vicinity of the adjacent coordinates in order to improve the resolution of the handwritten information each impose substantial processing, memory and communications bandwidth requirements, thereby disadvantageously increasing the cost and decreasing the efficiency of the pen-enabled computing device and communications channel. The imposition of additional memory, processing and communications bandwidth requirements is especially disadvantageous for pen-enabled computing devices, however, since many pen-enabled computing devices are hand-held devices, such as personal digital assistants (PDAs), personal communications systems (PCSs), or other hand-held computing devices, which are designed to be as small, lightweight, and generally inexpensive as practical. Moreover, if the processor and/or memory of the pen-enabled computing device are undersized, smoothing of the handwritten data and creation of new points in the vicinity of adjacent coordinates may require inordinately long periods of time, thereby somewhat restricting use of the penenabled computing device. Additionally, the user of the pen-enabled computing device may be unable to anticipate the manner in which the remote computing device will display or otherwise utilize the handwritten information, thereby preventing the handwritten information from being effectively smoothed or the resolution of the handwritten information from being appropriately adjusted prior to transmission to the remote computing device.
With the proliferation of e-mail, instant messaging, and other electronic communication techniques, a variety of different types of information are routinely transmitted between parties. For example, information such as text, pictures, drawings, links to web sites and the like, is routinely transmitted from one party to another. In order to transmit both the handwritten as well as the additional information to a remote computing device, either in a single message or in separate messages, the memory, processing and communications bandwidth requirements of the pen-enabled computing device are further disadvantageously increased. Moreover, if the additional information relates to or is otherwise associated with the handwritten information and is transmitted in another message, the recipient must appropriately associate or relate the handwritten information and the additional information that are delivered via separate messages. While a variety of conventional pen-enabled computing devices are available for capturing and processing handwritten information, it would be desirable for the images of the handwritten information that are eventually displayed to be smoothed and to have improved resolution without unnecessarily increasing the processing, memory and communications bandwidth requirements and, accordingly, the cost of the pen-enabled computing devices. Similarly, it would be desirable for a pen-enabled computing device to be capable of transmitting messages that include both handwritten information and some additional information without unnecessarily increasing the processing, memory and communications bandwidth requirements of the pen-enabled computing device.
The present invention therefore provides improved systems, computer program products and methods for transmitting and processing handwritten data in a manner that offers increased functionality by reducing the processing, memory and communications bandwidth requirements of the first computing device that captures the handwritten information. According to one aspect of the present invention, a system, computer program product and method are provided that smooth the handwritten information following the transmission of the handwritten data to a second computing device, thereby reducing the memory, processing, and communications bandwidth requirements of the first computing device. Similarly, according to another aspect of the present invention, a system, computer program product and method are provided for creating new points in the vicinity of at least some of the data points that comprise the handwritten information following the transmission of the handwritten data to a second computing device, thereby improving the resolution of the handwritten information without increasing the memory, processing and communications bandwidth requirements of the first computing device. According to yet another aspect of the present invention, a system, computer program product and method are provided that associate additional data with the handwritten information after the handwritten information has been transmitted from the first computing device that captured the handwritten information to a second computing device and which thereafter creates a composite comprised of both the handwritten information and the additional data, thereby reducing the processing, memory and communications bandwidth requirements of the first computing device and eliminating any need for the recipient to compile separate messages containing the handwritten information and any additional data.
According to one aspect of the present invention, a system, computer program product and method are provided for transmitting and processing handwritten data that smoothes the handwritten information following the transmission of the handwritten data from the first computing device that captured the handwritten information to a second computing device. According to this aspect of the present invention, the first computing device captures the handwritten information and subsequently transmits handwritten data representative of the handwritten information. Typically, the handwritten information is comprised of a plurality of data points. As such, the first computing device transmits handwritten data representative of the plurality of data points that comprise the handwritten information. According to this aspect of the present invention, a second computing device receives the handwritten data transmitted by the first computing device. The second computing device includes a processing element for rendering the handwritten information based upon the handwritten data. For example, the processing element can initially recover the handwritten information including the plurality of data points based upon the handwritten data that has been received. The processing element can then draw lines according to the plurality of data points in order to render the handwritten information.
Advantageously, the processing element of the second computing device of this aspect of the present invention is adapted to smooth the handwritten information while rendering the handwritten information following the transmission of the handwritten data to the second computing device. For example, the processing element can smooth the handwritten information by drawing smooth lines drawn according to the plurality of data points. In one common example, the processing element smoothes the handwritten information by applying an antialiasing algorithm to the handwritten information. The processing element can also be adapted to create a message that incorporates the smoothed handwritten information. The message may be an e-mail message, a facsimile message, an instant message or the like. In addition to or instead of transmitting a message including the smoothed handwritten information, the second computing device can include a memory device for storing the smoothed handwritten information.
In addition to a system including the first and second computing devices, a method and an associated computing program product for transmitting and processing handwritten data that includes smoothing the handwritten information following the transmission of the handwritten data to the second computing device are also similarly provided according to this aspect of the present invention. By smoothing the handwritten information, the aesthetic appearance of the handwritten information stored by the second computing device or embodied in a message created by the second computing device is improved. By smoothing the handwritten data at the second computing device following the transmission of the handwritten data, however, the system, the computer program product and the method of this aspect of the present invention reduce the memory, processing and communications bandwidth requirements of the first computing device that captured the handwritten information since the first computing device no longer is required to smooth the handwritten information.
According to another aspect of the present invention, a system, computer program product and method are provided for transmitting and processing handwritten data in which the resolution of the handwritten information is improved by creating new points based upon respective positions of at least some of the plurality of data points that comprise the handwritten information following the transmission of the handwritten data from the first computing device that captured the handwritten information to a second computing device. According to this aspect, the first computing device captures handwritten information comprised of a plurality of data points and transmits handwritten data representative of the handwritten information to a second computing device. The second computing device receives the handwritten data. The second computing device includes a processing element for rendering the handwritten information based upon the handwritten data. For example, the processing element initially recovers the handwritten information including the plurality of data points represented by the handwritten data prior to rendering the handwritten information. According to this aspect of the present invention, the processing element is adapted to create new points based upon respective positions of at least some of the plurality of data points, such as by means of cubic polynomial interpolation, approximation or curve-fitting, in order to better render the handwritten information following the transmission of the handwritten data to the second computing device. As such, the processing element can then render the handwritten information by drawing lines between the plurality of data points or the newly created points or both sets of points. After creating the new points to effectively improve resolution, the processing element can create a message that incorporates the handwritten information, such as an e-mail message, a facsimile message, an instant message or the like. In addition to or instead of transmitting a message including the handwritten information, the second computing device can include a memory device for storing the handwritten information following the creation of new points based upon respective positions of at least some of the plurality of data points.
In addition to a system that includes the first and second computing devices, a method and a corresponding computer program product for transmitting and processing handwritten data which creates new data points based upon respective positions of at least some of the plurality of data points of the handwritten information following the transmission of the handwritten data from the first computing device that captured the handwritten information to a second computing device are similarly provided according to this aspect of the present invention. By creating new points based upon respective positions of at least some of the data points, the resolution of the handwritten information stored by the second computing device or embodied in a message created by the second computing device is improved. By creating the new points at the second computing device following the transmission of the handwritten data, however, the system, the computer program product and the method of this aspect of the present invention reduce the memory, processing and communications bandwidth requirements of the first computing device that captured the handwritten information since the first computing device no longer is required to create new points based upon respective positions of at least some of the plurality of data points that comprise the handwritten information.
According to another aspect of the present invention, a system, a computer program product and a method are provided for transmitting and processing handwritten data that are capable of identifying additional data to be associated with handwritten information and that can create a composite comprised of both the handwritten information and the additional data. According to this aspect of the present invention, a first computing device captures the handwritten information and transmits handwritten data representative of the handwritten information. A second computing device receives the handwritten data transmitted by the first computing device. The second computing device includes a processing element for rendering the handwritten information based upon the handwritten data. Typically, the handwritten information is comprised of a plurality of data points such that the processing element initially recovers the handwritten information including the plurality of data points represented by the handwritten data and then draws smooth lines between the plurality of data points and/or the created points in order to render the handwritten information.
According to this aspect of the present invention, the processing element is also adapted to identify additional data to be associated with the handwritten information and to create a composite comprised of both the handwritten information and the additional data. Typically, instructions are provided regarding the additional data to be associated with the handwritten information such that the processing element can identify the additional data in accordance with the instructions. A variety of different types of data can be associated with the handwritten information such as predefined text, audio and/or video, a link to a web site, additional handwritten information, an image, a letterhead or the like. The handwritten information and the additional data can be processed in several manners. For example, the processing element may separately incorporate both the handwritten information and the additional data into a message in order to create the composite. Alternatively, the processing element may overlay either the handwritten information or the additional data upon the other one in order to create the composite. The processing element can then create a message such an e-mail message, a facsimile message, an instant message of the like, that incorporates the composite. In addition to or instead of creating a message that incorporates the composite, the second computing device can include a memory device for storing the handwritten information and the additional data.
In addition to the system for transmitting and processing handwritten data that identifies additional data to be associated with the handwritten information and that creates a composite including both the handwritten information and the additional data, a method and a corresponding computer program product are similarly provided. By identifying additional data to be associated with the handwritten information and creating a composite containing both the handwritten information and the additional data, the system, computer program product and method of this aspect of the present invention permit the memory, processing and communications bandwidth of the first computing device to be reduced. Additionally, the system, computer program product and method of this aspect of the present invention can provide both the handwritten information and the additional data to recipient in a convenient form without requiring the recipient to associate handwritten information and additional data that have been received via separate messages.
Accordingly, the systems, computer program products and methods of the present invention can provide improved functionality by smoothing the handwritten information, creating new points based upon respective positions of at least some of the data points that comprise the handwritten information in order to improve the resolution of the resulting image of the handwritten information and/or by associating additional data with the handwritten information and creating a composite containing both the handwritten information and the additional data according to various aspects of the present invention. By smoothing the handwritten information and creating the new points with the second computing device following the transmission of handwritten data representative of the handwritten information from the first computing device to the second computing device, the system, computer program product and method of these aspects of the present invention permit the memory, processing and communications bandwidth requirements of the first computing device to be reduced. Since the first computing device is oftentimes a handheld or portable computing device, reductions in the memory, processing and communications bandwidth requirements are particularly advantageous. For example, by smoothing the handwritten information and creating new points at the second computing device, the first computing device need not transmit as much handwritten data to the second computing device, thereby advantageously reducing the bandwidth required to transmit the handwritten data from the first computing device to the second computing device.